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Final  Report 

Implicit  memory,  perception  and  the  rapid  deployment  of  visual  attention  and  action 
Period  January  1, 2001  -  March  31, 2004. 


Executive  Summary 

The  issue  of  learning  and  memory  in  perception  and  in  the  deployment  of  visual  attention  was  examined  in  considerable 
detail,  with  7  of  the  14  papers  published  resulting  from  work  done  in  this  area.  This  included  an  investigation  learning  of 
the  deployment  of  visual  attention  itself  and  perceptual  learning  more  broadly.  Learning  of  the  deployment  of  transient 
attention  is  an  entirely  new  yet  robust  phenomenon.  It  is  mediated  by  a  unique  short  term  memory  system  which  is 
primitive,  which  learns  only  features  and  not  objects.  Yet  it  is  object  centered  and  not  retinotopic.  Furthermore  the 
learning  is  graded,  incremental  and  short  lived  no  more  than  30  seconds.  It  is  based  on  a  primitive  memory  kernel  which 
summates  linearly. 

We  have  also  discovered  with  respect  to  longer  term  perceptual  learning  that  very  short  naps  during  the  afternoon 
can  enhance  perceptual  learning  dramatically,  that  in  some  cases  beneficial  effects  are  as  good  as  a  whole  nights  sleep. 

We  suggest  that  for  a  nap  to  be  beneficial  it  must  contain  both  SWS  and  REM.  Afternoon  naps  fulfill  this  requirement.  In 
terms  of  a  neurophysiological  substrate,  the  learning  must  be  occurring  in  the  early  visual  areas  because  the  effects  are 
restricted  to  the  specific  visual  quadrants  stimulated. 

The  other  main  focus  of  the  work  has  been  the  role  of  learning  of  attentional  deployment  on  motor  behavior,  in 
particular  arm  movements.  In  this  area,  we  have  two  main  findings.  First  that  there  seems  to  be  a  direct  coupling  between 
the  deployment  of  focal  attention  and  action,  that  the  speed  of  motor  activities  is  determined  by  the  deployment  of  focal 
attention.  Second,  and  less  expected,  are  results  indicating  that  the  planning  and  execution  of  motor  actions  are  not 
separable.  Decisions  are  often  countermanded  by  mid-stream  changes  in  motor  actions.  This  can  occur  with  surprisingly 
little  cost  in  speed  or  accuracy  indicating  that  the  combined  brain  and  motor  plant  is  adept  at  parallel  processing  of 
simultaneous  motor  commands.  Since  the  physical  context  of  any  motor  act  is  dependent  dynamically  dependent  on  the 
state  of  the  motor  plant,  this  raises  the  question  as  to  how  successive  motor  acts  planned  in  parallel  are  executed.  Is  the 
physical  context  of  each  successive  motor  act  determined  purely  by  assessing  the  state  of  the  motor  plant  or  is 
feedforward  information  from  simultaneously  planned  motor  actions  also  relevant? 

Technical  Summary 


Learning  of  transient  attentional  deployment. 

The  main  effort  has  been  to  characterize  the  learning  of  transient  attentional  deployment,  an  unexplored  phenomenon  with 
broad  implications  for  perception  and  action.  Our  basic  findings  are  as  follows:  We  have  short  term  memory 
mechanisms  in  our  brain  which  enables  attention  to  go  more  efficiently  to  targets  attended  to  in  the  very  recent  past  (30 
seconds).  This  allows  attention  to  go  to  loci  in  the  environment  with  much  greater  speed.  The  memory  system  has  many 
striking  characteristics.  (1)  it  learns  very  quickly,  reaching  asymptotic  performance  within  4-8  attentional  fixations 
(Kristjansson  et  al,  2001).  (2)  it  is  restricted  to  just  one  kind  of  attentional  deployment,  a  very  fast  powerful  transient 
attentional  system  (Kristjansson  and  Nakayama,  2001).  (3)  the  system  is  involuntary,  it  cannot  be  countermanded  or 
overridden  (Kristjansson  et  al.,  2001).  (4)  it  is  very  primitive,  the  associations  are  to  features  and  not  objects.  The  memory 
system  for  example  cannot  speed  focal  attention  to  one  end  of  one  object  and  the  other  end  of  a  different  object.  Thus  it 
can  learn  only  to  go  to  simple  features  or  object  centered  loci  (right  or  left),  it  cannot  learn  about  objects  themselves 
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(Kristjansson  and  Nakayama,  2003  Nakayama  et  al.,  2004) .  (5)  the  memory  is  graded,  incremental  and  summates 
linearly.  It  can  be  described  in  terms  of  the  superposition  of  a  single  memory  kernel  function,  which  decays  exponentially 
over  time  (Nakayama  et  al,  2004) .  (6)  it  is  object  centered,  it  is  not  dependent  of  retinotopy  (Kristjansson  et  al.,  2001, 
Nakayama  et  al.,  2004).  (8)  the  system  represents  new  kind  of  associative  memory  system,  heretofore  unexplored , 
linking  the  act  of  attentional  deployment  to  features  in  the  visual  environment.  The  linkage  to  features  is  piecemeal  with 
multiple  associations  to  many  features  simultaneously  and  independently. 

The  existence  of  such  a  system  has  ramifications.  First,  it  has  implications  regarding  neural  function.  No  memory 
neurobiological  memory  systems  with  this  time  scale  have  been  identified  but  with  such  a  characteristic  signature,  we 
think  that  there  is  a  good  chance  that  a  neural  locus  can  be  identified.  This  could  have  the  twin  benefits  of  providing  a 
new  learning  preparation  to  examine  as  well  as  elucidating  the  network  characteristics  of  this  remarkably  robust  system. 

In  our  summary  chapter  (Nakayama  et  al.  2004  )  we  suggest  that  the  associative  learning  system  identified 
should  be  considered  more  precisely  as  modulating  in  the  short  term,  existing  associative  connections.  The  time  scale  of 
the  memory  process  suggests  the  possibility  of  qualitatively  different  underlying  mechanisms.  The  learning  curve,  while 
showing  the  typical  negative  accelerating  function  is  qualitatively  different  from  conventional  learning  processes  where 
incremental  learning  strength  is  proportional  to  the  remaining  learning  capacity  (Rescorla  and  Wagner,  1972).  In  the 
current  memory  system,  there  is  no  diminution  of  added  learning  strength,  the  negative  accelerating  learning  curve  results 
from  a  different  mechanism,  that  of  the  memory  kernel’s  exponential  decay.  Second,  the  existence  of  this  memory 
system  is  unusual  in  that  it  mediates  a  very  fast  response.  The  deployment  of  focal  attention  has  a  latency  short  extremely 
short,  below  50  milliseconds.  Thus  even  though  the  deployment  of  focal  attention  seems  reflex  like,  it  has  unsuspected 
flexibility. 

The  fact,  however,  that  the  memory  system  is  object  centered  but  does  not  leam  about  objects  is  puzzling.  As 
such,  the  neural  basis  of  this  form  of  plasticity  is  bound  to  be  revealing  about  underlying  neural  mechanisms.  Third,  the 
existence  of  this  system  is  bound  to  have  many  implications  regarding  function.  We  suggest  several.  The  system  provides 
an  alternative  to  the  widely  accepted  notion  of  “search  image  postulated  to  account  for  many  aspects  of  animal  foraging. 
Instead  of  a  search  image,  we  think  our  implicit  memory  system  can  account  for  the  foraging  literature  in  simpler 
mechanistic  terms.  In  addition,  we  think  that  the  memory  seen  portends  the  existence  of  similar  systems  mediating  a  wide 
range  of  cognitive  and  motor  functions.  For  example,  there  is  a  large  literature  on  task  switching  where  there  is  a  large 
penalty  paid  for  switching  any  aspect  of  a  repetitive  task.  Generalizing  our  memory  kernel  to  these  other  processes  could 
provide  a  simple  account  of  all  these  examples.  Finally  there  is  the  issue  of  “reinforcement”  or  reward,  what  enables  the 
system  to  strengthen  some  associations  and  not  others.  We  have  evidence  that  it  is  the  act  of  deploying  attention  in  a 
competitive  situation  that  is  of  importance.  Simply  directing  attention  to  a  single  target  for  example,  does  not  lead  that 
target’s  feature  to  become  strongly  associated  with  increased  speed  later  on. 


Naps  and  perceptual  learning 

Two  papers  which  comprised  Sara  Mednick’s  Ph.D.  dissertation  (Mednick  et  al.  2002,  2003),  show  an  important 
role  for  afternoon  naps  in  preventing  perceptual  deterioration  in  repeated  tasks  as  well  as  in  improving  subsequent 
performance.  The  task  was  a  texture  discrimination  task  where  the  observer  was  to  identify  the  orientation  of  an 
embedded  texture  within  a  larger  textured  field.  Our  first  important  finding  was  that  perception  in  a  repeated  tasks 
dramatically  deteriorated  during  the  day  as  a  consequence  of  over  practice.  We  then  found  that  this  deterioration  could 
be  completely  abolished  by  a  short  nap  in  the  afternoon.  In  a  second  paper,  we  examined  another  aspect  of  this  same 
phenomenon,  the  importance  of  short  afternoon  naps  in  facilitating  learning.  In  both  cases,  for  the  deterioration  and  for  the 
facilitation  of  learning,  the  beneficial  effects  of  a  nap  had  two  important  characteristics.  First,  the  effects  were 
retinotopic,  that  is  that  the  effect  of  the  nap  was  to  either  prevent  deterioration  or  improve  performance  only  within  a 
particular  quadrant  of  the  visual  field.  This  indicates  that  the  site  of  sleep  enhanced  learning  is  very  early  in  the  visual 
pathway,  in  those  areas  of  the  visual  cortex  that  have  preserved  retinotopic  maps.  Second  the  results  indicate  that  the 
effect  for  improvement  is  very  large,  a  short  nap  has  as  much  effect  in  perceptual  improvement  as  does  a  full  6  hours  over 
overnight  sleep. 

There  are  several  aspects  of  the  research  that  are  of  importance.  It  is  the  first  series  of  studies  to  show  that 
daytime  naps,  not  a  full  night  of  sleep  is  beneficial.  This  is  surprising  since  earlier  research  had  shown  that  sleeps  benefits 
would  accrue  only  if  there  were  6  hours  or  more  of  overnight  sleep.  We  argue  that  it  is  the  product  of  slow  wave  sleep 
(SWS)  and  rapid  eye  movement  sleep  (REM)  that  is  of  importance.  Thus  shorter  than  6  hours  of  overnight  sleep  is  not 


sufficient  because  of  insufficient  REM.  Afternoon  naps  are  beneficial  because  they  contain  significant  components  of 
both  SWS  and  REM.  Naps  earlier  in  the  morning  have  more  REM  and  later  afternoon  and  evening  naps  have  almost 
exclusively  SWS. 

Sara  Mednick  is  following  up  on  this  research  at  the  Salk  Institute,  finding  fMRI  correlates  of  the  learning  as  well 
as  showing  that  it  is  also  specific  to  particular  visual  features,  not  just  retinotopic  loci.  Taken  together,  it  suggests  a  new 
way  to  understand  the  consolidation  of  memories  in  early  visual  areas. 

Role  of  attention  in  motor  actions 

A  third  major  area  of  research  conducted  during  the  grant  period  was  an  investigation  of  the  role  of  attention  in 
the  guided  motor  planning  and  execution  of  hand  and  arm  movements.  This  project  has  been  conducted  in  collaboration 
with  graduate  student,  Joo  Hyun  Song.  Our  guiding  hypothesis  was  that  similar  to  its  role  in  the  initiation  of  eye 
movements,  motor  behavior  would  be  critically  dependent  of  the  deployment  of  focal  attention,  that  rapid  hand 
movements  would  occur  in  situations  where  focal  attention  was  most  effectively  directed.  To  investigate  these  issues  we 
essential  built  a  new  laboratory.  It  consisted  of  a  new  eye  movement  measuring  systems  and  two  method  to  measure  arm 
motions,  a  touch  screen  and  a  hand  tracker. 

In  our  first  series  of  studies  we  found  that  indeed,  for  every  case  where  a  particular  stimulus  manipulation  has 
been  shown  to  speed  the  deployment  of  visual  attention,  those  same  situations  also  speed  manual  hand  movements.  Thus, 
increasing  the  number  of  distractors  in  a  visual  search  task  (Bravo  and  Nakayama,  1994),  repeating  target  color  over  trials 
(Maljkovic  and  Nakayama,  1 994)  or  introducing  a  gap  between  the  fixation  offset  and  target  appearance  (Mackeben  and 
Nakayama,  1993)  all  resulted  in  speeded  arm  motions.  Detailed  measurements  of  the  hand  trajectories  indicate  that  these 
speeded  motor  acts  were  reactions  can  be  attributed  to  two  sources,  latency  to  the  initiation  of  the  movement  and  motor 
execution  duration. 

Following  up  on  our  previous  work  on  the  parallel  processing  of  saccadic  eye  movements  McPeek  et  al.,  2000), 
we  asked  whether  in  the  motor  programming  of  the  arm  with  such  a  huge  inertial  mass  in  comparison  to  the  eye  ball, 
whether  such  mid-flight  corrections  of  motor  actions  would  also  occur.  We  hypothesized  that  these  online  adjustment 
would  not  occur.  To  our  surprise,  we  found  strong  evidence  for  parallel  programming  of  arm  motions  and  that  despite  the 
strongly  curved  trajectories  accompanying  such  changes  in  direction,  little  was  sacrificed  in  terms  of  final  target  timing  or 
accuracy.  This  suggests  that  a  simple  serial  model,  with  appropriate  decisions  then  followed  sequentially  by  actions  is  not 
universally  applicable.  Instead,  wrong  initial  target  trajectories  followed  by  mid  flight  corrections  were  very  frequent  and 
had  very  little  cost,  either  in  speed  or  accuracy.  In  other  words,  if  there  is  a  choice  between  slow  accurate  decision 
making  followed  by  quick  direct  action  vs  a  much  faster  choice  and  error  correcting  highly  curved  complex  motor 
trajectories,  it  seems  that  our  system  can  opt  for  the  latter  in  substantial  numbers  of  cases  with  little  cost.  We  think  that 
the  prevalence  of  fast  decisions  and  large  scale  corrections  needs  much  more  investigation. 
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